AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	0.495855	2.68
	2_DecisionTree	Decision Tree	logloss	0.218947	12.32
	3_Default_Xgboost	Xgboost	logloss	0.198801	15.53
	4_Default_NeuralNetwork	Neural Network	logloss	0.22617	13.13
	5_Default_RandomForest	Random Forest	logloss	0.209468	19.78
the best	Ensemble	Ensemble	logloss	0.197531	1.92

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

10.7 seconds

Metric details

	score	threshold
logloss	0.218947	nan
auc	0.945282	nan
f1	0.944106	0.518876
accuracy	0.908639	0.518876
precision	0.99449	0.984508
recall	1	0.0392308
mcc	0.697582	0.518876

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.218947	nan
auc	0.945282	nan
f1	0.944106	0.518876
accuracy	0.908639	0.518876
precision	0.928183	0.518876
recall	0.960586	0.518876
mcc	0.697582	0.518876

Confusion matrix (at threshold=0.518876)

	Predicted as Major	Predicted as Minor
Labeled as Major	606	264
Labeled as Minor	140	3412

Learning curves

Decision Tree

Tree #1

Rules

if (APR Severity of Illness Code <= 2.5) and (Age Group <= 3.5) and (APR MDC Code > 0.5) then class: Minor (proba: 99.42%) | based on 7,536 samples

if (APR Severity of Illness Code > 2.5) and (APR Severity of Illness Description > 0.5) and (Age Group <= 3.5) then class: Minor (proba: 66.39%) | based on 1,693 samples

if (APR Severity of Illness Code <= 2.5) and (Age Group > 3.5) and (APR Severity of Illness Code > 1.5) then class: Minor (proba: 82.97%) | based on 1,374 samples

if (APR Severity of Illness Code > 2.5) and (APR Severity of Illness Description > 0.5) and (Age Group > 3.5) then class: Major (proba: 73.79%) | based on 1,339 samples

if (APR Severity of Illness Code > 2.5) and (APR Severity of Illness Description <= 0.5) and (CCS Diagnosis Code <= 253.5) then class: Major (proba: 95.64%) | based on 780 samples

if (APR Severity of Illness Code <= 2.5) and (Age Group > 3.5) and (APR Severity of Illness Code <= 1.5) then class: Minor (proba: 97.49%) | based on 517 samples

if (APR Severity of Illness Code > 2.5) and (APR Severity of Illness Description <= 0.5) and (CCS Diagnosis Code > 253.5) then class: Major (proba: 64.0%) | based on 25 samples

if (APR Severity of Illness Code <= 2.5) and (Age Group <= 3.5) and (APR MDC Code <= 0.5) then class: Major (proba: 100.0%) | based on 1 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

11.7 seconds

Metric details

	score	threshold
logloss	0.22617	nan
auc	0.949591	nan
f1	0.941589	0.574694
accuracy	0.905925	0.574694
precision	1	0.999991
recall	1	3.97683e-06
mcc	0.700077	0.574694

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.22617	nan
auc	0.949591	nan
f1	0.941589	0.574694
accuracy	0.905925	0.574694
precision	0.939216	0.574694
recall	0.943975	0.574694
mcc	0.700077	0.574694

Confusion matrix (at threshold=0.574694)

	Predicted as Major	Predicted as Minor
Labeled as Major	653	217
Labeled as Minor	199	3353

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

18.1 seconds

Metric details

	score	threshold
logloss	0.209468	nan
auc	0.95759	nan
f1	0.944106	0.509799
accuracy	0.908639	0.509799
precision	1	0.994556
recall	1	0.0222841
mcc	0.708145	0.568183

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.209468	nan
auc	0.95759	nan
f1	0.944106	0.509799
accuracy	0.908639	0.509799
precision	0.928183	0.509799
recall	0.960586	0.509799
mcc	0.697582	0.509799

Confusion matrix (at threshold=0.509799)

	Predicted as Major	Predicted as Minor
Labeled as Major	606	264
Labeled as Minor	140	3412

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

13.8 seconds

Metric details

	score	threshold
logloss	0.198801	nan
auc	0.960648	nan
f1	0.945364	0.445778
accuracy	0.912709	0.595654
precision	1	0.996271
recall	1	0.00182651
mcc	0.735056	0.595654

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.198801	nan
auc	0.960648	nan
f1	0.944999	0.595654
accuracy	0.912709	0.595654
precision	0.956722	0.595654
recall	0.933559	0.595654
mcc	0.735056	0.595654

Confusion matrix (at threshold=0.595654)

	Predicted as Major	Predicted as Minor
Labeled as Major	720	150
Labeled as Minor	236	3316

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

1.6 seconds

Metric details

	score	threshold
logloss	0.495855	nan
auc	0.5	nan
f1	0.890895	0.72285
accuracy	0.803256	0.72285
precision	0.803256	0.72285
recall	1	0.72285
mcc	0	0.72285

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.495855	nan
auc	0.5	nan
f1	0.890895	0.72285
accuracy	0.803256	0.72285
precision	0.803256	0.72285
recall	1	0.72285
mcc	0	0.72285

Confusion matrix (at threshold=0.72285)

	Predicted as Major	Predicted as Minor
Labeled as Major	0	870
Labeled as Minor	0	3552

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	3
4_Default_NeuralNetwork	1
5_Default_RandomForest	1

Metric details

	score	threshold
logloss	0.197531	nan
auc	0.961377	nan
f1	0.945995	0.510743
accuracy	0.912935	0.514193
precision	1	0.991164
recall	1	0.00598748
mcc	0.734224	0.593923

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.197531	nan
auc	0.961377	nan
f1	0.945965	0.514193
accuracy	0.912935	0.514193
precision	0.943185	0.514193
recall	0.948761	0.514193
mcc	0.722091	0.514193

Confusion matrix (at threshold=0.514193)

	Predicted as Major	Predicted as Minor
Labeled as Major	667	203
Labeled as Minor	182	3370

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back